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ABSTRACT 
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Accounting Data Entry (APADE) system, in a before/after quasi-experimental 
design that measured outputs (workload, productivity), inputs (staff size, staff 
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sick leave, and leave without pay) using archival data. While workload decreased , 
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implementation. This result was obtained as the size of the staff decreased and 
overtime usuage declined sharply. The implementation of APADE streamlined the 
document process significantly at the test site. 
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INTRODUCTION 



I . 

Computer industry literature claims that no longer is 
automation synonymous with automatic productivity 
improvement. Organizations are now struggling with 
developing methods to measure productivity improvement in 
the white collar sector. The Federal Government has 
invested millions of dollars in the development of automated 
procurement systems. However, little has been done with 
regards to the measurement of change in productivity 
resulting from the implementation of these systems. 

Our intent in this study is to further develop a 
methodology which can be used to measure productivity for 
automated procurement systems. An installed Automation of 
Procurement and Accounting Data Entry (APADE) system is 
examined to assess the impacts of office automation on 
productivity. This system has been in operation for over 
two years at a Navy supply center, which we call NSC Duarte. 

APADE was designed to provide the Navy field contracting 
system with a standard automated procurement mechanism. It 
has been synonymous with automated procurement in the 
Department of the Navy since the 1970' s. Prior to the 
current installation of APADE, NSC Duarte was using a system 
called APADE II. APADE II was essentially a manager's 
inforroation system which provided management statistics to 
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supervisors upon request. For the purpose of this study, 
and for the duration of this thesis, the term "pre APADE" 
will be used when referring to APADE II. 

The current system implementation is called APADE and it 
includes many of the functions contained in pre APADE. 

APADE was designed as a decision support system for Navy 
buyers. Functions such as providing a price history and 
source file allow the buyer access to on-line information 
that had previously only been available on a manual basis. 
The system also provides an automatic document preparation 
capability, removing the need to manually generate each 
document. In addition, APADE provides automated dociament 
control and tracking status of procurement actions . APADE 
provides real-time contracting information to management 
while allowing selective access to appropriate customers. 
Although APADE can now process large contracts, this thesis 
will concentrate on contracts under $ 25 , 000 , procured by the 
Small Purchasing Branch (SPB) at Duarte on the APADE 
system. A description of the procurement processes under 
both APADE and pre APADE is provided in Appendix A. 

Our goal is to measure specific organizational factors 
to determine what effect APADE had on the effectiveness of 
Duarte's Small Purchasing Branch. In doing this, we used 
the ratio of outputs divided by inputs to calculate 
productivity. Productivity is enhanced as the proportion of 
outputs to inputs increases. 
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In this study, the inputs used include staff size, grade 
structure, and overtime hours worked. (See Appendix B) The 
outputs are the number of purchase orders processed and 
Procurement Action Lead Time (PALT) measured in days. This 
study also examines social factors which affect 
productivity: annual leave taken, sick leave used, and leave 
without pay taken. (See Appendix C) Lastly, we evaluate 
productivity as a measure of purchase orders processed per 
procurement labor hour. (See Appendix D) ^ 



^Appendixes B, C, and D contain summary data per pay period. 
For the comprehensive sample of raw data used by this study contact 
Professor William J. Haga at the Naval Postgraduate School, 
Monterey, California. 
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II . 



LITERATURE REVIEW 



Throughout the 1980' s industry has attempted to automate 
the white collar environment. It had been assumed that 
productivity improvement would come with automation. 

However, this may not have been the case and many 
corporations are now questioning office automation projects 
more thoroughly than before. 

A. WHAT IS NOT HERE 

This study does not attempt to consider the measurement 
of productivity of knowledge workers (professional, 
technical, administrative, and managerial) . The focus of 
our study concerns the measurement of productivity of 
clerical workers. 

B. BASIC INPUT/OUTPUT MEASUREMENT 

There are numerous definitions that have been applied to 
productivity. Some are vague, "Productivity is a particular 
type of behavior within an organization. It may be high, a 
positive and desiraible behavior/ or low, a negative and 
undesirable behavior." (Edosomwan, 1987, p. 5) Others are 
relatively specific "total productivity is a ratio of all 
measureQjle outputs to the sum of all measuraible inputs." 
(Edosomwan, 1987, p. 5) However, the basic form that most 
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definitions follow is that productivity is the ratio of some 
output to some input . 

Bain (1982) indicates that it is not just a simple ratio 
nor is it a measure of production or output produced. "It 
is a measure of how well resources are combined and utilized 
to accomplish specific, desirable results." (Bain, 1982, 
p. 3) 

C. IMMEASURABLE PRODUCTIVITY 

Measuring productivity in a white collar environment has 
long been considered difficult at best. Rowe states that 
"the great nemesis of measuring white collar output has been 
the inability to quantify the end results of the white 
collar employee." (Rowe, 1981, p. 43) Even the U. S. 
Department of Labor states that white collar productivity is 
so difficult to measure that they only put out that 
statistic by industries (Leeke, 1988) . 

Much of the productivity to be gained from white collar 
workers is of an intangible /immeasuradDle variety. Leeke 
(1988) states it is frequently a sxabjective measurement, 
often not based on quantif iaJDle characteristics. One can 
easily see the difficulty in measuring such unquantif iable 
factors as improved customer service, improved employee 
morale, and timely information needed for decision making 
(Barclift and Linson, 1988) . Schwartz (1987) states the if 
there is no reduction in headcount there are no easy means 
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available for measuring white collar productivity. In fact, 
the "payoffs become more difficult to assess as information 
technologies - office systems, personal computers, expert 
systems, and application enhancement projects - are 
increasingly directed at improving the performance of white 
collar workers." (Schwartz, 1987, p. 47) 

D. SUBSTITUTING ATTITUDE SURVEYS FOR INPUT/OUTPUT 
Another form of measurement that has been used to 

measure productivity as well as system performance, usage, 
and effectiveness is the user attitude survey. Miller and 
Doyle (1987) indicate that there have been a number of 
instances where the measurement of user satisfaction has 
been used as a surrogate for overall effectiveness of 
information systems within organizations.lt has been 
generally accepted within the academic MIS community that 
user satisfaction can be correlated to information usage and 
system success 'Bailey and Pearson, 1983) . 

E. VARIETIES OF INPUT/OUTPUT ANALYSIS 

There are a number of ways in which productivity 
improvements may be achieved. Bain (1982) provides a 
summary: 

1. Output increases while input decreases. 

2. Output remains the same while input decreases. 

3. Output increases while input remains the same. 
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To this list. Sink (1985) adds two other ways in which 
productivity improvement may be obtained: 

1. Output increases while input increases at a slower 
rate . 

2. Output decreases while input decreases at a greater 
rate . 

In measuring productivity, Bain (1982) states that one 
must consider the interplay between various factors within 
an organization. "While the output may be related to many 
different inputs in the form of productivity ratios, each of 
the separate productivity ratios is influenced by a 
combination of many relevant factors." (Bain, 1982, p. 3) 

The influencing factors to be considered range from the 
quality and availability of materials to the attitude and 
skill level of the work force. 

In his description of productivity measures. Sink 
defines two basic categories of productivity measures. The 
first, static productivity ratios, involve output divided by 
input for a given period of time. The second category, 
dynamic productivity ratios, are essentially particular 
static productivity ratios at one point in time divided by 
the same ratio at some prior period. This provides us with 
an index which reflects the change in productivity between 
different periods in time. Within each category there are 
three different types of productivity measures. The only 
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differentiating characteristic between them is the degree of 
input that is used in the denominator of the equation. 
Partial- factor measures include only one class of input, 
such as labor, capital, or energy. Multi-factor measures 
include more than one class, while total- factor measures 
include all classes of input. (Sink, 1985) 

F. API; ADMINISTRATIVE PRODUCTIVITY INDEX 

Bolte (1983) and his colleagues at Intel wished to 
develop a quantitative system which would focus on improving 
administrative productivity as well as on reducing head 
count. In searching the literature they found that there was 
very little practical information available on 
quantitatively controlling headcount growth and measuring 
and improving administrative productivity. As a result, 
they set out to develop a quantitative system for measuring 
productivity in administrative functions. 

To begin with, Bolte used the definition of productivity 
currently in use at Intel: output divided by input. 
Specifically, the physical units of output are divided by 
the total nvimber of hours it takes to produce them. It was 
decided not to use financial measures of input or output, 
such as sales or revenues, because their "definition is 
understandable and controllable at the line management 
level, which is where productivity improvements must take 
place." (Bolte, 1983, p. 47) 
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Next, a set of indicators was developed which were 
affected by their measurement systems. The first was to 
establish a set of quantity and quality indicators. Each 
department would establish a goal for the quality and 
quantity of its work. Second the quality and quantity 
indicators of one function or site would be compared with 
those of another. Lastly, the ratio between supervisors and 
employees is employed as a measure of the ratio between 
direct labor to indirect labor (supervisors) . 

In an effort to develop a quantitative system for 
measuring productivity, Bolte conceptualized administrative 
areas as paper processing factories "with specific inputs 
and required outputs, much like an assembly line, so that 
production techniques can be applied." (Bolte, 1983, p. 48) 
With this in mind he developed an Administrative 
Productivity Index (API) which would provide a means of 
comparing productivity against an estciblished base line at 
different points in time. API is calculated by dividing 
output by labor hours, the result is expressed in hours per 
unit (HPU) . The output measures must be both physical and 
countable, while input is all of the hours of work paid for 
by the organization, less vacation, aibsenteeism, and sick 
leave, during the period which the output was generated. 

Once baseline API values have been established 
productivity may be measured over time by comparing the API 
figures collected over a period. Productivity enhancements 
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are then obtained through simplifying tasks, applying 
workload management techniques, and monitoring the API. The 
labor requirement of a unit to accomplish its mission is 
reduced, which reduces headcount and thus improves 
productivity . 

G. CONSENSUS MODEL: INFERRED OUTPUT 

A consensus model seeks to obtain agreement among 
managers on the projected inferred benefits to be obtained 
through the implementation of a specific computer system. 
Schwartz (1907) states that the inferred output technique is 
an alternative that may be used to measure productivity when 
a direct output model cannot be developed. Managers first 
determine estimates of the value of various tasks. This 
information coupled with their reasoning is then pooled. A 
consensus is then formed after repeated estimates and 
sharing of information. The underlying assumption of this 
model is that an increase in output will generate an 
increase in profit. This increased profit can then be used 
to infer an increase in productivity. General Telephone and 
Electronics (GTE) has used a consensus model in its 
operations . The "consensus models are most appropriate for 
judging potential payoff when there is little quantitative 
basis for making estimates of value." (Schwartz, 1987, p. 
48) 
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H. INFERRED INPUT MODEL 



Inferred input models "use projected increases in 
efficiency and effectiveness among workers rather than 
actual, verified cuts in labor or head count." (Schwartz, 
1987, p. 48) Generally, the projections generated are based 
on a task/time matrix that indicates the time that people 
spend on tasks and the time savings offered by the 
implementation of the computer system. The Times- 
Savings/Times Salary (TSTS) model being used by IBM is based 
on research done by Booze, Allen, and Hamilton, Inc. People 
first determine the amount of tim.e they expend on activities 
such as reading and typing. Any savings that are gained 
through the implementation of a specific computer technology 
are then multiplied by each employee's salary. 

The TSTS model is simple and easy to use, making it a 
popular model. However, there are drawbacks. Poppel (1982) 
contends that the TSTS model counts time saved on lower 
value activities, such as the work of a clerk, equivalently 
with higher value activities. In other words, TSTS cannot 
distinguish between making a white-collar worker a better 
manager or analyst and making that worker a better clerk or 
receptionist. (Schwartz, 1987) 

I. WORK VALUE ANALYSIS 

Schwartz (1987) and his colleagues have developed a 
hybrid model which they named the Work Value Analysis (WVA) 
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model. This model not only evaluates the efficiency of white 
collar workers but also provides input on their 
effectiveness. Efficiency is the increased work accomplished 
in a period of time, while effectiveness refers to doing the 
right things . This incorporates the thought that the members 
of an organization do not spend all of their time at work 
performing primary activities. A large portion of their 
time includes performing supporting activities, such as 
clerical duties. 

The WVA model uses two forms of productivity 
improvement : 

1 . Technology can shorten the amount of time required to 
complete a given task or it can allow more of the task 
to be completed in the seime eimount of time. 

2. Technology can be the basis for a shift in a work 
pattern that allows more time to be spent on primary 
activities and less on lower valued activities such as 
support, clerical, lost time. (Schwartz, 1987, p.52) 

The first form identifies improvements in efficiency, 
while the second form accounts for improvements in the 
effectiveness of an organization. By identifying its 
primary or high value activities, an organization can target 
those activities for improvement with the greatest 
return/payoff. 

Schwartz contends that its strength is in its "objective 
determination of payoff when external dollar criteria 
relating to profit of value of work, other than salary. 
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cannot be measured or otherwise inferred.” (Schwartz^ 1987^ 
p. 52) . However, WVA requires a significant investment in 
time and effort to implement and the mathematical 
computations can become complex, 

J. COST DISPLACEMENT MODEL 

Schwartz (1987) notes that direct input models, such as 
the cost displacement model, are useful when inputs can be 
determined and outputs cannot be measured. In order to 
account for output, the cost displacement model assumes that 
output either remains the same or increases. Therefore, if 
output remains the same and inputs decrease, such as number 
of workers, there is an improvement in productivity. 

The largest plus for cost displacement models is that 
they are simple to use. The only requirement is that real 
labor cuts or eqpjipment savings be achieved due to the 
introduction of new information systems. However, without a 
cut in labor or equipment savings cost, displacement models 
are inappropriate , 

K. NPMM: NORMATIVE PRODUCTIVITY MEASUREMENT METHODOLOGY 

The normative productivity measurement methodology 
(NPMM) seeks to obtain various output/input ratios through a 
consensus building method called the nominal group technique 
(NGM) , Sink describes NPMM as an ’’action-research, 
involvement-participative, organizational , development- 
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oriented approach to measuring productivity." (Sink, 1985, 
p. 139) 

The process is encapsulated in five stages. The first 
stage uses the normative group technique to generate a 
prioritized list of the measures, ratios, or indices of 
productivity for each of the corresponding organizational 
units. The second stage involves converting those measures 
into a workable productivity measurement system. This task 
is usually referred to productivity analysts for 
development. The third stage includes briefings, reviews, 
discussions, and revisions until a workaJDle system is 
drafted. The fourth stage integrates this new system with 
other performance measurement and control systems already 
implemented within the organization. Stage five involves 
the continuous monitoring and feedback of information based 
on the previously determined ratios. (Sink, 1985) 

NPMM allows the participants to formulate the measures 
to be used in the measurement system. The inclusion of 
organizational members provides the advantage of 
establishing commitment cunong them which is important is any 
project is to succeed. 

L. MFPHM: MULTIFACTOF. PRODUCTIVITY MEASUREMENT MODEL 

Multifactor productivity measurement (MFPMM) is a 
consultative, data base/accounting system oriented 
methodology which collects its data from system 
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documentation rather than people. It is diagnostic in a 
passive, absolute, and objective sense as opposed to an 
active, relative, and subjective sense (Sink, 1985) . In 
using the various accounting data available within an 
organization a series of base indices is generated which may 
be compared with the values in the succeeding periods 
determine relative productivity improvements/declines. 

The multifactor productivity measurement model can be 
used to provide information on a variety organizational 
areas. These include: 

1. Obtain an overall, integrated measure of productivity 
for the firm. 

2. Assess and evaluate bottom-line results of specific 
productivity improvement efforts. 

3. Monitor the historical productivity performance and 
measure how much, in dollars, profits were affected by 
productivity growth or decline. 

4. Assist with setting productivity objectives and general 
strategic planning in areas such marketing and cost 
management. (Sink, 1985) 

MFPMM and other measurement systems similar to it are 
currently being used in industry. For instance, 

Westinghouse has used a similar methodology when measuring 
white collar productivity. Each department would establish 
its own measurement criteria or performance ratios. This 
information would then be used to calculate a composite 
index, which is calculated by assigning weights to the 
measures and then combining the values into a composite 
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value (Rowe, 1981) . These base indexed measures can then be 
used to compare the relative change in productivity in 
succeeding periods. 

M . PRODUCTIVITY MAP 

The productivity map by Pacesetter Software has been 
developed to assist managers in establishing and measuring 
productivity goals of white collar workers. It consists of 
a multi-step process which establishes and tracks four 
primary measures of productivity: quantity, quality, 
timeliness, and cost. (Miller, 1987) 

Step one involves answering questions on the 
characteristics and goals of the next level up in the 
organization. "Productivity map then prepares an overview 
report that summarizes your department's mission, strategic 
direction, and critical success factors based on these 
answers." (Miller, 1987, p. 43) Questions are then answered 
regarding your department's role within the organization as 
well as your perceptions of what the customers' needs are. 
These are then compared with the customers' needs and 
relative priorities. The final step involves the 
development of productivity measures which are then used 
with the assistance of the program's database, graphics, and 
report generation abilities to track the productivity of the 
unit . 
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N. TOTP (TASK-ORIENTED TOTAL PRODUCTIVITY MEASUREMENT) 

A total productivity measure uses the total output with 
respect to the total input. The Task-Oriented Total 
Productivity Measurement Model (TOTP) considers all possible 
measurable input and output components. The measures used 
in this model are developed into a matrix which can be used 
to measure changes in productivity. 

There are several steps involved in the process. A 
formal study of the work environment, processes, and 
procedures is first conducted. The results are then shared 
with those individuals that will be affected by the 
implementation. The next step involves the selection of the 
measurement criteria to be used. Overhead expenses are then 
allocated among the units on either a "proportional basis" 
or through a "complexity factor. " A base period is selected 
which acts as a reference period to compare the measurement 
criteria. Next, various forms and instrviments are developed 
to capture the input and output elements to be measured. 
Personnel are then trained in the measuring processes and 
data collection begins. Trend analysis and interpretation of 
the findings can then begin. (Edosomwan, 1987) 

There are several advantages to be obtained by using the 
TOTP model : 

1. The indices can be used in the productivity planning 
and improvement phases. 
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2. The productivity indices derived offer flexibility in 
that they may change as the task parameters, resources 
used, etc. change. 

3. The indices are comparable over time and can be used to 
measure the productivity of various organizational 
elements, such as task, work groups, and departments. 

(Edosomwan, 1987) 

However, Bain (1982) indicates that while total factor 
productivity makes conceptual sense, "it is extremely 
difficult to identify and capture all related inputs for any 
component output of the organization." (Bain, 1982, p. 56) 

O . OBJECTIVES MATRIX 

Many organizations have found that productivity 
measurement can be quite difficult and complex. In fact, 
far too few organizations actively pursue effective 
productivity measurement. The Oregon Productivity Center 
(OPC) has developed what it believes is an easy-to-implement 
methodology based on its Objectives Matrix. The Objectives 
Matrix combines all of an operation's important productivity 
criteria into one interrelated format, thus overcoming the 
problem of complexity. 

The first step to identify the key performance 
indicators. Once this is done each indicator is to be 
thoroughly defined and sources of this information 
identified. Performance is then assessed over time and the 
information obtained is used in determining productivity 
objectives. Next, each indicator is assigned an importance 
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weighting, usually 100 total points distributed across the 
indicators (Felix and Riggs, 1983) . Improvement /decline in 
productivity over time is then measured against the initial 
index . 

Felix and Riggs (1983) indicate that there are several 
advantages obtained by using the Objectives Matrix: 



1. Measures are normalized by establishing a uniform 
quantitative rating system. 

2. Many dimensions of performance may be followed at the 
same time providing a global perspective of the 
organization . 

3. The Objectives Matrix focuses on results rather than on 
activities . 



P. SUMMARY 

A review of the availaible literature revealed the 
following: 



1. There is little documentation availcible regarding 
productivity measurement using measurements prior to an 
automation implementation and after. 

2. There is little documentation regarding the measurement 
of productivity improvement due to the automation of 
office work. 

3. The use of job satisfaction surveys as an indicator of 
productivity rather than input/output . 

4. Although there is still a belief cunong many that 
computerization of office work leads to productivity 
enhancement, there are numerous examples which indicate 
that this view is changing towards one requiring actual 
justification. 
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Q . OBJECTIVE 

The preceeding methodologies are based on a common 
mechanism for the measurement of productivity, which they 
define as a ratio of outputs to inputs in one form or 
another. This study will also use this ratio model in the 
calculation of productivity. 

This thesis will: 

1 . Develop a methodology for productivity measurement that 
is based on a before/after quasi-experimental design. 

2. Conduct an office automation study to test the 
methodology . 

3. Base the study on quantifiable measures of input and 
output . 
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Ill . 



METHODOLOGY 



A. CONDUCT OF THE STUDY 

1 . Prelude to the S 2 unple 

The investigators sampled data from a military 
procurement organization: the Purchasing Department at the 
Naval Supply Center (NSC) Duarte, South Dakota. The 
selection of this site is due to its having an operational 
APADE installation and the availability of pre-APADE 
records. This study has provided an opportunity to measure 
productivity changes since the implementation of APADE at 
NSC Duarte. 

After initial discussions with the staff, it was 
determined that the primary data needed to conduct this 
study would be availa±»le. We decided on a 'before and 
after' data collection design to compare productivity 
statistics prior to and after the implementation of APADE. 
Data sampled prior to the implementation of APADE were 
actual purchase orders and reports from the on-site 
archives . Data for after implementation was acquired from 
on-site reports as well as documentation generated by APADE. 

2 . A Description of APADE 

The APADE configuration studied is a small purchase, 
computer-based system designed to provide a Navy field 
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contracting office with a standard automated procurement 
mechanism. It's capabilities include: 

1. Automated documentation preparation. 

2 . Automated document control and tracking status . 

3. Procurement management information. 

4. Buyer support information. 

This system provides real-time contracting 
information to management while allowing selective access by 
customers. Additional information on APADE is presented in 
Appendix B . 

3 . Experimental Design Development 

Our study employed a quasi-experimental methodology 
(Campbell and Stanley, 1968) . Samples were taken for a 
twelve month period before and a twelve month period after 
APADE implementation. It was determined that three types of 
data would be used to evaluate productivity: 

1 . Inputs . 

2 . Outputs . 

3. Social Effects. 

a . Inputs 

These include the measurement of cjuantifiedDle 
elements, such as grade structure, size of staff, and 
overtime worked. 
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b. Outputs 



Various factors that are the product of work done 
include Procurement Action Lead Time (PALT) , number of 
purchase orders processed per unit time, and the number of 
labor hours per purchase order. PALT is the time it takes 
to process a purchase order. The PALT counter begins upon 
the receipt of a purchase action request by the Small 
Purchase Branch and stops when a purchase order has been 
awarded. The number of purchase orders processed during a 
year is used to indicate the volume of work, while the 
number of labor hours needed to perform a single procurement 
is a measure of productivity, 
c. Socxa.1 Effects 

Social effects act as surrogate indicators to the 
work environment. This study used annual leave, sick leave, 
and leave without pay. 

The aunount of annual leave used is determined by 
management's capability to allow personnel to expend leave. 
If workload is at a level to where authorizing leave is not 
detrimental to the organization, leave may be granted. 
Conversely, if there are significant backlogs in the 
workload, management may not have the flexibility to allow 
personnel to take leave on a discretionary basis. 

Sick leave policy provides a way for personnel 
who are unhappy or stressed to avoid the demands of the work 
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place. Sick leave is earned with the decision to use it 
made exclusively by the employee. Since no documentation is 
required from a physician to confirm an employee's status, 
it is an available escape mechanism for an employee who is 
dissatisfied with the work environment or workload. 

Leave Without Pay (LWOP) is similar to annual 
leave. The exception is that employees who use LWOP do not 
receive any monetary compensation while in a LWOP status . 

An employee would use LWOP as a last resort when they do not 
have annual leave or sick leave time "on the books." 

4 . Analysis Strategy 

The statistical test selected to evaluate the data 
is the difference of means in a "Student's t-distribution" 
test. A series of null hypotheses are tested for 
rejection/inability to reject, using the t-test for 
statistical significance. 

5 . Collection of Data 

Pre-APADE data were collected from records 
maintained in storage for the fiscal year 1987. The data, 
obtained by manually sampling the physical records, were 
used to establish the baseline for the activity's PALT. 

PALT data for the period after APADE installation were 
obtained by sampling listings generated by the system. 
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B. THE COLLECTION DESIGN 

The Regional Contracting Department is made up of two 
divisions, which in turn are made up of various branches. 
This study concerns itself with the data obtained for the 
Small Purchase Branch, which is currently staffed with 58 
people, primarily female civilian employees. 

The collection of data from the Contracting Department 
archives involved sampling purchase orders before and after 
APADE installation. The following describes how records 
used for estimating PALT were sampled. 

1 . Before APADE Installation 

The population consisted of 85479 records for fiscal 
year 1987 purchase orders. This population figure was 
obtained from the Monthly Procurement Summary of Actions 
Report (DD Form 1057) . These transactions were made up of 
either a single requisition or multiple requisitions for 
similar material grouped together from a single customer. 

In sampling the purchase orders, we randomly 
selected records from departmental archives. A total of 532 
purchase orders were sampled from the period prior to APADE 
being implemented. 

2 . After APADE Installation 

The process for scimpling post implementation 
purchase orders is the same except that the samples were 
taken from listings generated by APADE. Our population 
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consisted of 59905 records for fiscal year 1989 purchase 
orders obtained from the DD Form 1057. A total of 637 
purchase orders were sampled from the period after APADE had 
been implemented. 

3 . Additional Measures 

Our study also required that we retrieve archival 
data from other sources as well. Data on the use of annual 
leave, sick leave, leave without pay, and overtime hours 
worked was obtained from NSC Duarte Comptroller Department's 
records and reports. Information regarding the number of 
personnel employed and grade structure within the Purchasing 
Division was also obtained from the command' s Comptroller 
Department . 

A framework was used to represent the interaction of 
inputs and outputs, prior to and after the implementation of 
APADE at NSC Duarte. (See Table I) It provides a means to 
test the null hypothesis for the measures listed. In 
testing a null hypothesis, we are attempting to determine if 
the data sampled before and after this phase of APADE 
implementation were statistically the Scime. The alternative 
hypothesis is that the before and after data are different. 

C. INSTRUMENTATION 

One method to measure a change in productivity is by 
analyzing the ratio of inputs to outputs. In a system, a 
certain amount of input is required to produce some output. 
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TABLE I. 



PRODUCTIVITY MATRIX 



MEASURES 


PRE APADE 


APADE 


INPUTS 


staff Size 
Grade Structure 
Overtime 


staff Size 
Grade Structure 
Overtime 


OUTPUTS 


No. Purchase 

Orders 

PALT 


No. Purchase 

Orders 

PALT 


SOCIAL 

EFFECTS 


Annual Leave 
Sick Leave 
Leave Without Pay 


Annual Leave 
Sick Leave 
Leave Without Pay 


PRODUCTIVITY 

MEASUREMENT 
1 


Purchase Orders 
per Labor Hour 


Purchase Orders 
per Labor Hour 



If the system is changed so as to require fewer inputs or to 
produce more output with the same inputs, then productivity 
is enhanced. 

1 . Inputs 

There are several inputs to the requisitioning 
process that were used in this evaluation: 

1 . Before and after mean annual number of employees . 

2. Before and after mean GS level. 

3. Before and after mean overtime worked. 
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The measurement of the mean annual number of 



employees provided a figure of the average number of 
personnel required by the Small Purchase Branch to perform 
its function. The mean GS level of the staff provided an 
indicator of the level of expertise to perform the functions 
of the unit. The mean overtime worked represented the extra 
time spent, beyond regular working hours, to complete the 
work assigned. 

2 . Outputs 

The output of the procurement process contained two 
aspects which were measured. The first considered the 
difference between the n\imber of requisitions processed 
prior to and after implementation. The second measured the 
PALT both before and after implementation. 

PALT has been selected as our Productivity Measure 
of Effectiveness, because it provides a measure of 
productivity. PALT represents the time it takes to process 
a purchase order once a requisition has been received by the 
purchasing division. A typical scenario would begin with 
the submission of a requisition. After proceeding through 
the Customer Service and Technical Branches, the requisition 
is determined to be a purchase item rather than something 
that is available through the supply system. The PALT timer 
begins when a requisition is received by the Purchase 
Division. Once the processing has been completed and the 
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requisition has been awarded to a vendor, the PALT timer 
stops. The difference between the award date and the date 
of receipt of the requisition, plus one day, is the PALT for 
a purchase.^ 

The quantity parameter considered the volume of work 
performed by the Small Purchase Branch over a unit of time. 
In measuring quantity we used the average number of purchase 
orders processed each month, 

3 . Social Effects 

In previous studies which performed similar analyses 
(Barclift and Linson, 1988) it was noted that various social 
factors affected unit performance. Reduced workload, 
increased tecimwork, and improvements in training were 
indicators that were mentioned. The social effects that 
were measured include: 

1. Before and after mean annual leave taken. 

2. Before and after mean sick leave taken. 

3. Before and after mean leave without pay used. 

Although these social factors are not directly measures of 
input or output, they are objective indicators of 
satisfaction obtained from the working environment. 



^One day is added so that PALT would reflect the inclusive 
dates of receipt of the requisition and the award date of the 
contract . 
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4 . Productivity Measurement 

The productivity measurement used in this study is 
simply the number of transactions processed/completed per 
labor hour. Labor hours were calculated as the total number 
of hours worked by both purchasing personnel (buyers) and 
administrative support personnel . This factor provided us 
with a direct measurement of the productivity of the 
purchasing and administrative support personnel. 

D. ANALYSIS STRATEGY 
1 . Measurements 

A difference of means test was used to compare the 
various input, output, social factors, and productivity 
ratio prior to and after the implement at i or of APADE. Null 
hypotheses stated that there was no change in the various 
measurements as a result of APADE' s implementation. 

A total of 1169 purchase orders were sampled. Of 
these, 532 records represented the PALT prior to 
implementation, while 637 records were collected for the 
post implementation period. 

The command' s Comptroller Department provided access 
to staff size, grade and step level, annual leave taken, 
sick leave used, and leave taken without pay data from on- 
hand NAVCOMPT FORM 206 records for both periods. Staff size 
was verified via the small purchase department's point of 
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contact. Overtime data were also obtained from the 
command's comptroller departmental records. 

2. Choosing the Appropriate Statistical Test 

When testing hypotheses using two populations, with 
the samples being either dependent or independent of one 
another, and having an unknown population standard deviation 
(<7) , the appropriate method would be a student's t- 
distribution (Porter and Hamm, 1986) . We used the t-test 
to assess differences between the means of various 
indicators, both before and after APADE implementation, for 
samples taken independently from the two populations. 

The reason for taking independent samples from each 
population is that the pool of employees within the division 
was not identical before and after implementation. There 
was the usual turnover of personnel that organizations 
endure and a restructuring of the unit because of APADE. 

The null hypothesis (H„) stated that the two 
populations are statistically the ssune, while the 
alternative hypothesis (H.) stated that the before measure 
is greater or less than the after measure to a statistically 
significant degree. 

Using MINITAB Fundamental/RW Software for Students 
(Anderson/Eynon) , a one tailed t-distribution test (TTEST) 
was selected because we were predicting direction. The 
corresponding significance levels are shown in Table II. We 
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have chosen the .01 significance level as the appropriate 
level at which to test all measures collected. 



TABLE II. T-DISTRIBUTION SIGNIFICANCE LEVELS 



Significance Level 


t-value 


.05 


1.645 


.01 


2.326 
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IV. 



FINDINGS 



A. INPUTS 

Measures of inputs to the Small Purchase Branch of the 
Regional Contracting Department located at NSC Duarte, were 
collected and summarized in the ensuing categories: 

1. Size of the staff. 

2. Grade structure (GS level). 

3. Overtime worked. 

These factors were collected for time periods before and 
after the implementation of APADE. 

1 . Before APADE 

a . Staff Size 

The size of the staff was calculated as the mean 
size of the staff on hand for each pay period at the Small 
Purchasing Branch (SPB) . The mean size of the staff before 
the implementation of APADE was 73.4 personnel with a 
standard deviation of 3.5. This represents the period from 
January 1, 1987 through December 31, 1987. Data were 
collected from NAVCOMPT FORM 206' s maintained by the 
command's comptroller department. 

b. Grade Structure 

The grade level of the staff was computed as the 
average GS level for all personnel at the Small Purchasing 
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Branch (SPB) per pay period. The mean grade level (GS) of 
the staff per pay period was GS 5.66, with a standard 
deviation of 0.098. This represents the period from January 
1, 1987 through December 31, 1987. Data were obtained from 

worker' s Service Card (NAVCOMPT FORM 206) maintained by the 
command's comptroller department, 
c . Overtime 

Overtime worked by SPB personnel was calculated 
as the mean number of overtime hours worked by the entire 
staff for each two-week pay period for the year prior to the 
implementation of APADE. The average overtime worked per 
two-week pay period was 519.5 hours, with a standard 
deviation of 306.0. The mean overtime worked per worker per 
two-week pay period was 6.7 hours, with a standard deviation 
of 7.1. The mean overtime worked per buyer in the SPB was 
177.3 hours annually, with a standard deviation of 145.6. 

The mean overtime worked per administrative support worker 
was 128.9 hours, with a standard deviation of 121.6. The 
annual mean overtime worked by all SPB personnel was 162.7 
hours per worker, with a standard deviation of 139.9. 

2. After APADE In^lementation 
a . Staff Size 

The mean size of the staff after the 
implementation of APADE was 62.4 personnel, with a standard 
deviation of 1.24. This represents the period from January 
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1, 1989 through December 31, 1989. Data were collected from 
NAVCOMPT FORM 206' s maintained by the command's comptroller 
department • 

Jb. Grade Structure 

The mean grade level (GS) of the staff per pay 
period was GS 6.06, with a standard deviation of 0.006. 

This represents the period from January 1, 1989 through 

December 31, 1989. Data were obtained from worker's Service 
Card (NAVCOMPT FORM 206) maintained by the command's 
comptroller department, 
c . Overtime 

The average overtime worked per two-week pay 
period was 124.4 hours, with a standard deviation of 168.9. 
The mean overtime worked per worker per two-week pay period 
was 2.0 hours, with a standard deviation of 2.7. The 
average amount of overtime worked per buyer in the SPB was 
58.1 hours annually, with a standard deviation of 71.7. The 
average cimount of overtime worked annually per 
administrative support worker was 25.2 hours, with a 
standard deviation of 33.5. The average amount of overtime 
worked annually by all SPB personnel was 51.3 hours per 
worker with a standard deviation of 66.8. 
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3. Testing the Null Hypotheses for Inputs 
a . Staff Size 

The null hypothesis (H^,) concerning the size of 
the staff at the Small Purchasing Branch (SPB) stated that 
the size of the staff per two-week pay period after the 
implementation of APADE (X) was statistically equal to that 
prior to implementation (X^,) , X=Xq) . The alternative 

hypothesis (H.) stated that the size of the staff per two- 
week pay period after the implementation (X) is less than 
the size prior to the implementation (X^,) , (H, : X<Xo) . 

Upon analysis, the t-distribution test score for 
this measure was -44.97 at the 0.0000 significance level 
(left-tailed test) . Since this is less than the t- 
distribution baseline for this study (t-value -2.326 at the 
0.01 significance level), the null hypothesis (H^) was 
rejected. Therefore, the alternative hypothesis (H,) was 
accepted; mean staff size at the SPB per two-week pay period 
after the implementation of APADE (X) was statistically less 
than the mean staff size prior to implementation (X^,) . 

T 2 ±>le III represents the number of buyers and 
administrative support personnel immediately before the 
implementation of APADE and at the end of first full year of 
APADE operation. 
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TABLE III. EMPLOYEES 





PRE APADE 


APADE 


BUYERS 


58 


50 


ADMINISTRATIVE 

SUPPORT 


25 


13 



b. Grad0 Structure 

The null hypothesis (H„) of the grade structure 
of the staff at the SPB stated that the grade level of the 
staff per two-week pay period after the implementation (X) 
of APADE was statistically equal to that prior to 
implementation (X„) : (Ho; X=X„) . The alternative 

hypothesis (H.) stated that the grade level of the staff per 
two-week pay period after the implementation (X) is greater 
than the level prior to the implementation (X^) , (H, : X>X^) . 

Upon analysis, the t-distribution test score for 
this measure was 370.15 at the 0.0000 significance level 
(right-tailed test) . Since this is greater than the t- 
distribution baseline for this study (t-value 2.326 at the 
0.01 significance level), the null hypothesis (H„) was 
rejected. Therefore, the alternative hypothesis (H.) was 
accepted; mean grade level at the SPB per two-week pay 
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period after the implementation of APADE (X) was 
statistically greater than the mean staff level prior to 
implementation (X^) . 

c. Overtime 

The first null hypothesis (H^) concerning 
overtime stated that the overtime worked by all personnel 
per two-week pay period after the implementation of APADE 
(X) is statistically equal to that worked prior to 
implementation (X„) , (H^: X=X„) . The alternative hypothesis 

(H.) stated that the overtime worked by all personnel per 
two-week pay period after the implementation (X) is less 
than that worked prior to the implementation (X„) , (H. : 

X<XJ . 

Analysis finds that the t-distribution test score 
for this measure is -11.93 at the 0.0000 significance level 
(left-tailed test) . Since this is less than the t- 
distribution baseline for this study (t-value 
-2.326 at the 0.01 significance level), the null hypothesis 
(Hg) was rejected. Therefore, the alternative hypothesis 
(H.) was accepted; mean overtime worked by all personnel per 
two-week pay period after the implementation of APADE (X) 
was statistically less than the mean overtime worked prior 
to implementation (X^) . 

The second null hypothesis (H„) for overtime 
worked stated that the overtime worked per worker per two- 
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week pay period after the implementation of APADE (X) is 
statistically equal to that worked prior to implementation 
(X^) , (H„: X=X^) . The alternative hypothesis (H.) stated 

that the overtime worked per worker per two-week pay period 
after the implementation (X) is less than that worked prior 
to the implementation (X„) , (H, : X<X„) . 

Upon analysis, the t-distribution test score for 
this measure was -14.06 at the 0.0000 significance level 
(left-tailed test) . Since this is less than the t- 
distribution baseline for this study (t-value -2.326 at the 
0.01 significance level), the null hypothesis (H^,) was 
rejected. Therefore, the alternative hypothesis (H.) was 
accepted; mean overtime worked per worker per two-week pay 
period after the implementation of APADE (X) was 
statistically less than the mean overtime worked prior to 
implementation (X^) . 

Tcibles IV and V represent a summary of the total 
cimount of overtime worked before and after implementation. 
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TABLE IV. PRE APADE OVERTIME 



TYPE 


OVERTIME 


NO. OF 


MEAN 


PERSONNEL 




PERIODS 


OVERTIME 


BUYERS 


10285.0 


26 


177.3 


ADMIN 


3221.3 


26 


128.9 


TOTAL 


13506.3 


26 


162.7 



TABLE V. APADE OVERTIME 



TYPE 


OVERTIME 


NO. OF 


MEAN 


PERSONNEL 




PERIODS 


OVERTIME 


BUYERS 


2906.2 


26 


58.1 


ADMIN 


327.8 


26 


25.2 


TOTAL 


3234.0 


26 


51.3 



B . OUTPUTS 

Measures of outputs from the Small Purchase Branch of 
the Regional Contracting Department at NSC Duarte were 
collected and summarized in the ensuing categories : 

1 . Number of purchase orders . 

2. Procurement Action Lead Time (PALT) . 

These factors were collected for time periods before and 
after the implementation of APADE. 
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1 . Before APADE 



a. NumbBr of Purchase Orders 

The number of purchase requests processed is 
calculated as the number of purchase orders completed for 
each pay period for the year prior to the implementation of 
APADE. The average number of purchase requests processed 
per pay period by the SPB before the implementation of APADE 
was 3287.7, with a standard deviation of 655.1. This 
represents both single and multiple item procurements for 
fiscal year 1987. Data were collected from the Procurement 
Summary of Actions Report (DD FOPH 1057) that was manually 
generated on site. 

b. Procurement Action Lead Time (PALT) 

PALT is computed as the average number of days 
required to complete a purchase request, from the date 
received by the SPB to the date the contract for the 
purchase request is awarded to a contractor, plus one day.^ 
PALT prior to the implementation of APADE was 52.4 days, 
with a standard deviation of 33.8. Data were collected from 
a saimpling of the actual purchase order records that were 
retained on site. See Table VI for a summary of the samples 
taken prior to the implementation of APADE. 



^The one day is added so that PALT will represent the number 
of days between the receipt date and award date, inclusive. 
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2 . After APADE Is^lementation 
a . Number of Purchase Orders 

The average number of purchase requests processed 
per pay period by the SPB after the implementation of APADE 
was 2304.0, with a standard deviation of 454.4. This 
represents both single and multiple item procurements for 
fiscal year 1989 (after the implementation of APADE) . Data 
were collected from the Procurement Summary of Actions 
Report (DD FORM 1057) that was manually generated on site. 

Jb. Procurement Action Lead Time (PALT) 

PALT after the implementation of APADE was 23.5 
days, with a standard deviation of 24.7. Data were 
collected from a scuapling of the actual purchase order 
records that were retained on site. See Table VI for a 
summary of the scimples taken after the implementation of 
APADE. 



TABLE VI. SAMPLE BREAKDOWN BY ISSUE GROUP 





ISSUE GROUP 


ISSUE GROUP 


ISSUE GROUP 


PERIOD 


1 


2 


3 


PRE APADE 


69 


260 


203 


(FY 1987) 


12.97% 


48.87% 


38.16% 


APADE 


118 


333 


186 


(FY 1989) 


18.52% 


52.28% 


29.20% 
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3. Testing the Null Hypotheses for Outputs 
a. Number of Purchase Orders 

The null hypothesis (H„) concerning the number of 
purchase orders stated that the mean number of purchase 
orders processed per pay period after the implementation of 
APADE (X) is statistically equal to the number processed per 
pay period prior to implementation (X„) , X=X^) . The 

alternative hypothesis (H.) stated that the mean number of 
purchase orders processed per pay period (X) is less than 
the number processed per pay period prior to implementation 
(XJ, (H.: X<XJ . 

Upon analysis, the t-distribution test score for 
this measure was -11.04 at the 0.0000 significance level 
(left-tailed test) . Since this is less than the t- 
distribution baseline for this study (t-value 
-2.326 at the 0.01 significance level), the null hypothesis 
was rejected. Therefore, the alternative hypothesis (H.) 
was accepted; the mean nvimber of purchase orders processed 
per pay period after the implementation of APADE (X) was 
statistically less than that prior to implementation (X„) . 

Jb. Procurement Action Lead Time (PALT) 

The null hypothesis (H^) concerning the average 
PALT stated that the mean value obtained after the 
implementation of APADE (X) is statistically equal to the 
value obtained prior to implementation (X^) , (H^: X=X„) . The 
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alternative hypothesis (H,) stated that the mean PALT value 
(X) is less than the value obtained prior to implementation 
(XJ , (H.: X<X„) . 

Upon analysis, the t-distribution test score for 
this measure was -29.49 at the 0.0000 significance level 
(left-tailed test) . Since this is less than the t- 
distribution baseline for this study (t-value -2.326 at the 
0.01 significance level), the null hypothesis (H„) was 
rejected. Therefore, the alternative hypothesis (H.) was 
accepted; the mean PALT after the implementation of APADE 
(X) was statistically less than that prior to implementation 
(X„) . Tcibles VII and VIII represent the summary of 
Procurement Action Lead Time (PALT) for the before and after 
periods . 



TABLE VII. PALT SUMMARY BY ISSUE GROUP 





SAMPLE MEAN 




STANDARD 














DEVIATION 




PERIOD 


IG 


IG 


IG 


IG 


IG 




IG 




1 


2 


3 


1 


2 




3 


PRE APADE 


48.07 


55.72 


50.21 


35.85 


35.97 




30.62 


(FY 1987) 
















APADE 
(FY 1989) 


11.20 


23.98 


30.52 


18.29 


24.11 


26.44 
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TABLE VIII. 



PALT SUMMARY BY PERIOD 



PERIOD 


SAMPLE 

MEAN 


STANDARD 

DEVIATION 


PRE APADE 
(FY-1987) 


52.398 


33.792 


APADE 

(FY-1989) 


23.52 


24.713 



C . SOCIAL EFFECTS 

Measures of social effects at the Small Purchase Branch 
(SPB) of the Regional Contracting Department at NSC Duarte, 
were collected and summarized in the ensuing categories: 

1. Amount of annual leave taken. 

2. Amount of sick leave used. 

3. Amount of leave without pay utilized. 

These factors were collected for time periods before and 
after the implementation of APADE. 

1 . Before APADE 

a. Annual Leave 

The cimount of annual leave used by SPB personnel 
was calculated as the mean number of hours used by all 
buyers and administrative support personnel for each two- 
week pay period for the year before the implementation of 
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APADE. The average amount of annual leave used per two-week 
pay period for the SPB staff was 442.2 hours, with a 
standard deviation of 202.5. The average amount of annual 
leave used per worker per two-week pay period was 5.4 hours, 
with a standard deviation of 2.2. The average amount of 
annual leave used per buyer in the SPB was 152.9 hours 
annually, with a standard deviation of 52.0. The average 
amount of annual leave used per administrative support 
worker was 105.1 hours annually, with a standard deviation 
of 50.9. The amount of annual leave used by all SPB 
personnel was 138.5 hours per worker annually, with a 
standard deviation of 55.9. See Table IX for a summary 
breakdown by personnel type for the period prior to 
implementation of APADE. 
b. Sick ItoavB 

The amount of sick leave used by SPB personnel 
was calculated as the mean number of hours used by all 
buyers and administrative support personnel for each two- 
week pay period for the year before the implementation of 
APADE. The average eunount of sick leave used per two-week 
pay period for the SPB staff was 255.8 hours, with a 
standard deviation of 69.5. The average amount of sick 
leave used per worker per two-week pay period was 3.2 hours, 
with a standard deviation of 1.5. The average amount of 
sick leave used annually per buyer in the SPB was 82.2 



46 



hours, with a standard deviation of 36.4. The average 
amount used per administrative support worker 75.3 hours 
annually, with a standard deviation of 46.5. The average 
amount of sick leave used by all SPB personnel was 80.1 
hours per worker annually, with a standard deviation of 
39.6. See Table IX for a summary breakdown by personnel 
type for the period prior to implementation of APADE. 



TABLE IX. PRE APADE SOCIAL EFFECTS 



TYPE 


ANNUAL 

LEAVE 


SICK 

LEAVE 


LEAVE 

WITHOUT 

PAY 

(LWOP) 


MEAN 

ANNUAL 

LEAVE 


MEAN 

SICK 

LEAVE 


MEAN 

LWOP 


BUYERS 


8870 


4768 


3243 


152.9 


82.2 


55.9 


ADMIN 


2628 


1883 


796 


105.1 


75.3 


31.9 


OVERALL 


11498 


6651 


4039 


138.5 


80.1 


48.7 



c. Leave Without Pay 

The amount of leave without pay used by SPB 
personnel was calculated as the mean number of hours used by 
all buyers and administrative support personnel for each 
two-week pay period for the year before the implementation 
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of APADE. The average amount of leave without pay used per 
two-week pay period for the SPB staff was 155.4 hours, with 
a standard deviation of 77.3. The average cimount of leave 
without pay used per worker per two-week pay period was 1 . 9 
hours, with a standard deviation of 5.6. The average amount 
of leave without pay used per buyer in the SPB was 55.9 
hours annually, with a standard deviation of 171.0. The 
average amount of leave without pay used annually per 
administrative support worker was 31.9 hours, with a 
standard deviation of 51.0. The average amount of leave 
without pay used by all SPB personnel was 48.7 hours per 
worker annually, with a standard deviation of 145.6. See 
Tcible IX for a sijmmary breakdown by personnel type for the 
period prior to implementation of APADE. 

2 . After APADE Inclement at ion 
a. Annual Deave 

The average amount of annual leave used per two- 
week pay period for the SPB staff was 380.1 hours, with a 
standard deviation of 152.5. The average amount of annual 
leave used per worker per two-week pay period was 6.1 hours, 
with a standard deviation of 2.0. The average cunount of 
annual leave used per buyer in the SPB was 158.4 hours 
annually, with a standard deviation of 63.8. The average 
cunount of annual leave used per administrative support 
worker was 151.0 hours annually, with a standard deviation 
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of 27.9. The amount of annual leave used by all SPB 
personnel was 156.8 hours per worker annually, with a 
standard deviation of 48.8. See Table X for a summary 
breakdown by personnel type for the period after the 
implementation of APADE. 

b. Sick Leave 

The average amount of sick leave used per two- 
week pay period for the SPB staff was 262.0 hours, with a 
standard deviation of 78.3. The average amount of sick 
leave used per worker per two-week pay period was 4.2 hours, 
with a standard deviation of 2.4. The average amount of 
sick leave used annually per buyer in the SPB was 105.3 
hours, with a standard deviation of 63.8. The average 
amount used per administrative support worker was 119.3 
hours annually, with a standard deviation of 54.2. The 
average amount of sick leave used by all SPB personnel was 
108.2 hours per worker annually, with a standard deviation 
of 61.8. See Table X for a summary breakdown by personnel 
type for the period after the implementation of APADE. 

c. Leave Without Pay 

The average amount of leave without pay used per 
two-week pay period for the SPB staff was 112.1 hours, with 
a standard deviation of 65.5. The average amount of leave 
without pay used per worker per two-week pay period was 1.8 
hours, with a standard deviation of 4.8. The average amount 
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of leave without pay used per buyer in the SPB was 48.8 
hours annually, with a standard deviation of 136.0. The 
average amount of leave without pay used annually per 
administrative support worker was 36.6 hours, with a 
standard deviation of 52.1. The average amount of leave 
without pay used annually by all SPB personnel was 46.3 
hours per worker, with a standard deviation of 123.2. See 
Table X for a summary breakdown by personnel type for the 
period after the implementation of APADE. 



TABLE X. APADE SOCIAL EFFECTS 



TYPE 

PERSONNEL 


ANNUAL 

LEAVE 


SICK 

LEAVE 


LEAVE 

WITHOUT 

PAY 

(LWOP) 


MEAN 

ANNUAL 

LEAVE 


MEAN 

SICK 

LEAVE 


MEAN 

LWOP 


BUYERS 


7918 


5262 


2439 


158.4 


105.3 


48.8 


ADMIN 


1963 


1551 


475 


151.0 


119.3 


36.6 


TOTAL 


9881 


6813 


2914 


156.8 


108.2 


46.3 
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3 . Testing the Null Hypothesis for Social Effects 
a . Annual Leave 

The first null hypothesis (H„) concerning annual 
leave stated that the amount of leave taken by all workers 
per two-week pay period after the implementation of APADE 
(X) is statistically equal to that prior to implementation 
(X„) ; (H„: X=X„) . The alternative hypothesis (H.) stated 

that amount of leave taken by all workers per two-week pay 
period after the implementation (X) is less than that amount 
taken prior to the implementation (X^) , (H, : X<X^) . 

Upon analysis, the t-distribution test score for 
this measure was -2.08 at the 0.024 significance level 
(left-tailed test) . Since this is not less than the t- 
distribution baseline for this study (t-value -2.326 at the 
0.01 significance level), the null hypothesis was not 
rejected. Therefore, the mean annual leave taken by all 
personnel per two-week pay period after the implementation 
of APADE (X) was statistically equal to that prior to 
implementation. 

The second null hypothesis (H^) concerning annual 
leave stated that the amount of leave taken per worker per 
two-week pay period after the implementation of APADE (X) is 
statistically equal to that taken prior to implementation 
(X^) , (H^; X=X^,) . The alternative hypothesis (H.) stated 

that the amount of leave taken per worker per two-week pay 
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period after the implementation (X) is more than that amount 
taken prior to the implementation (X^) , (H, : X>X„) . 

Upon analysis, the t-distribution test score for 
this measure was 2.90 at the 0.0026 significance level 
(right-tailed test) . Since this is greater than the t- 
distribution baseline for this study (t-value 2.326 at the 
0.01 significance level), the null hypothesis (H^) was 
rejected. Therefore, the alternative hypothesis (H,) was 
accepted; the mean annual leave taken per worker per two- 
week pay period after the implementation of APADE (X) was 
statistically greater than that prior to implementation 
(XJ . 

b. Sick LBavB 

The first null hypothesis (H^,) concerning sick 
leave stated that the mean amount of sick leave taken by all 
workers per two-week pay period after the implementation of 
APADE (X) is statistically equal to that prior to 
implementation (X^) , (Hq: X=X^) . The alternative hypothesis 
(H,) stated that the mean amount of sick leave taken by all 
workers per two-week pay period after the implementation (X) 
is greater than that amount taken prior to the 
implementation (X^) , (H, : X>X„) . 

Upon analysis, the t-distribution test score for 
this measure was 0.40 at the 0.34 significance level (right- 
tailed test) . Since this is not greater than the t- 
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distribution baseline for this study (t-value 2.326 at the 
0.01 significance level), the null hypothesis (H„) could not 
be rejected. Therefore, the mean amount of sick leave taken 
by all personnel per two-week pay period after the 
implementation of APADE (X) was statistically equal to that 
prior to implementation (X„) . 

The second null hypothesis (H^) concerning sick 
leave stated that the mean amount of sick leave taken per 
worker per two-week pay period after the implementation of 
APADE (X) is statistically equal to that taken prior to 
implementation (X„) , (H„: X=X„) . The alternative hypothesis 

(H^) stated that the mean amount of sick leave taken per 
worker per two-week pay period after the implementation (X) 
is more than that amount taken prior to the implementation 
(XJ, (H.: X>XJ . 

Upon analysis, the t-distribution test score for 
this measure was 3.50 at the 0.0009 significance level 
(right-tailed test) . Since this is greater than the t- 
distribution baseline for this study (t-value 2.326 at the 
0.01 significance level), the null hypothesis (H^) was 
rejected. Therefore, the alternative hypothesis (H,) was 
accepted; the mean amount of sick leave taken per worker per 
two-week pay period after the implementation of APADE (X) 
was statistically greater than that prior to implementation 
(XJ . 
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c. Loava Without Pay 

One of the null hypotheses (H^) concerning leave 
without pay stated that the mean amount of leave without pay 
taken by all workers per two-week pay period after the 
implementation of APADE (X) is statistically equal to that 
prior to implementation (X„) , (H„: X=X„) . The alternative 

hypothesis (H,) stated that the mean amount of leave without 
pay taken by all workers per two-week pay period after the 
implementation (X) is less than that amount taken prior to 
the implementation (X„) , (H, : X<X„) . 

Upon analysis, the t-distribution test score for 
this measure was -3.37 at the 0.001 significance level 
(left-tailed test) . Since this is less than the t- 
distribution baseline for this study (t-value 
-2.326 at the 0.01 significance level), the null hypothesis 
(H^) was rejected. Therefore, the alternative hypothesis 
(H,) was accepted; the mean amount of leave without pay 
taken by all personnel per two-week pay period after the 
implementation of APADE (X) was statistically less than that 
prior to implementation (X^) . 

The other null hypothesis (H^) concerning leave 
without pay stated that the mean eunount of leave without pay 
taken per worker per two-week pay period after the 
implementation of APADE (X) is statistically equal to that 
taken prior to implementation (X^) , (H^: X=X„) . The 

alternative hypothesis (H.) stated that the mean eimount of 



54 



leave without pay taken per worker per two-week pay period 
after the implementation (X) is less than that amount taken 
prior to the implementation (X^) , (H. : X<Xj,) . 

Upon analysis, the t-distribution test score for 
this measure was -0.12 at the 0.55 significance level (left- 
tailed test) . Since this is not less than the t- 
distribution baseline for this study (t-value -2.326 at the 

0.01 significance level), the null hypothesis (Ho) was not 
rejected. Therefore, the mean amount of leave without pay 
taken per worker per two-week pay period after the 
implementation of APADE (X) was statistically equal to that 
prior to implementation (X^) . 

D . PRODUCTIVITY MEASUREMENT 

1. Before APADE 

The number of purchase orders processed per labor 
hour is the measurement of the number of purchase orders 
processed during the year divided by the number of actual 
labor hours accrued during the year. The mean number of 
purchase orders processed per Icibor hour prior to the 
implementation of APADE was 0.5564 purchase orders per hour, 
with a standard deviation of 0.1019. 

2 . After APADE Implementation 

The mean number of purchase orders processed per 
labor hour after the implementation of APADE was .5236 per 
hour, with a standard deviation of .0882. The data were 
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collected from both the Procurement S\immary of Actions 
Report (DD FORM 1057) that was manually generated on site 
and the SPB personnel' Service Card (NAVCOMPT FORM 206 - 
MECHANIZED) . 

3. Testing the Null Hypothesis for Productivity 

The null hypothesis concerning the productivity 
measurement at the SPB stated that the number of purchase 
orders processed per labor hour after the implementation of 
APADE (X) was statistically equal to that prior to 
implementation X=X^) . The alternative hypothesis 

stated that the nvimber of purchase orders processed per 
labor hour after the implementation is less them that prior 
to the implementation (H, : X<X^) . 

Upon analysis, the t-distribution test score for 
this measure was -1.90 at the 0.035 significance level 
(left-tailed test) . Since this is greater than the t- 
distribution baseline for this study (t-value -2.326 at the 
0.01 significance level), the null hypothesis is not 
rejected. The productivity measurement at the SPB after the 
implementation of APADE (X) was statistically equal to that 
prior to implementation. 

E. SUMMARY OF FINDINGS 

A summary of T-Test results of the inputs, outputs, 
productivity measurement, and social effects are presented 
in tcibles XI, XII, XIII, and XIV respectively. 
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TABLE XI. 



SUMMARY OF T-TEST RESULTS: INPUTS 



INPUTS 


PRE-APADE 


APADE 


T-TEST 

RESULTS 


MEAN STAFF SIZE 


73.35 


62.42 


T = -44.97 


Std. Deviation 


3.48 


1.24 


p = 0.0000 


GRADE LEVEL 


5.66 


6.06 


T = 370.15 


Std. Deviation 


.098 


. 006 


p = 0.0000 


MEAN OVERTIME 








(per 2-week pay 


519.46 


124.383 


T = -11.93 


period) Std. Dev. 


306.03 


168.894 


p = 0.0000 


MEAN OVERTIME 








(per worker per 2- 
week period) 


6.7229 


2.0270 


T = -14.06 


Std. Dev. 


7.0831 


2.6508 


p = 0.0000 



TABLE XII. SUMMARY OF T-TEST RESULTS; OUTPUTS 



OUTPUTS 


PRE-APADE 


APADE 


T-TEST 
RESULTS 1 


1 MEAN PURCHASE 
; REQUESTS PROCESSED 


3287.7 


2304.0 


T = -11.04 

p = 0.0000 


Std. Dev. 


655.1 


454.4 


1 


MEAN PROCUREMENT 
ACTION LEAD TIME 


52.4 


23.5 


T = -29.49 

p ~ 0.0000 


Std. Dev. 


33.8 


24.7 
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TABLE XIII 



PRODUCTIVITY MEASUREMENT T-TEST RESULTS 



PRODUCTIVITY 

MEASUREMENT 


PRE-APADE 


APADE 


T-TEST 

RESULTS 


MEAN PURCHASE 
REQUESTS PROCESSED 
PER LABOR HOUR 


0.5564 


0.5236 


T = -1.90 
p = 0.035 


standard Deviation 


0.1019 


0.0882 





TABLE XIV. SUMMARY OF T-TEST RESULTS: SOCIAL EFFECTS 



SOCIAL EFFECTS 


PRE-APADE 


APADE 


T-TEST 

RESULTS 


MEAN ANNUAL LEAVE 








(per 2-week pay 


442.22 


380.05 


T = -2.08 


period) 
Std. Dev. 


202.46 


152.47 


p = 0.024 


MEAN ANNUAL LEAVE 








(per worker per 2- 


5.3995 


6.1203 


T = 2.90 


week period) 
Std. Dev. 


2.1938 


1.9751 


p = 0.0026 


MEAN SICK LEAVE 








(per 2-week pay 


255.83 


262.03 


T = 0.40 


period) 
Std. Dev. 


69.459 


78.313 


p « 0.34 


MEAN SICK LEAVE 








(per worker per 2- 


3.1539 


4.2018 


T = 3.50 


week period) 
Std. Dev. 


1.5326 


2.3732 


p = 0.0009 


MEAN LEAVE WITHOUT 








PAY (per 2 -week 


155.40 


112.05 


T = -3.37 


pay period) 
Std. Dev. 


77.29 


65.536 


p « 0.0012 


MEAN LEAVE WITHOUT 








PAY (per worker 


1.8717 


1.8003 


T = -0.12 


per 2-week period) 
Std. Dev. 


5.6010 


4.7991 


p = 0.55 
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V. 



ANALYSIS AND CONCLUSION 



A. INPUTS 

1. Staff Size 

The mean staff size prior to the implementation of 
APADE was 73.4 personnel. The mean staff size after 
implementation was 62.4 personnel. This represents a 
decrease of 15 percent in the number of personnel in the 
Small Purchasing Branch (SPB) at NSC Duarte. With the 
remaining factors being held constant, a decrease in 
headcount would provide a decreased input measure . Combined 
with a constant output measure, by definition, using the 
ratio of outputs to inputs, this by itself would provide an 
increase in productivity. 

The amount of administrative support required prior 
to implementation was not needed after APADE' s 
implementation. This is due to APADE' s design allowing 
buyers to enter the data into the system as events occur 
rather than transporting documents to data entry clerks for 
input. This decreases the number of administrative support 
personnel required for support . Although a decrease in 
staff size by itself would act to increase productivity the 
implementation of APADE is not the only factor which may 
have had this effect. The number of purchase orders 
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submitted to NSC Duarte had also decreased. Management 
sources indicate that the decrease in the number of purchase 
requests received had no effect on the decrease in the 
number of personnel between the pre and post APADE periods. 
However, because regression analysis was beyond the scope of 
this study we did not address the relative significance of 
these two factors. Therefore, it can not be categorically 
stated that the decrease in staff size was in whole or part 
due to the implementation of APADE. 

2 . Grade Structure 

The mean grade level prior to the implementation of 
APADE was GS 5.66. The mean grade level measured after 
implementation was GS 6.06. This represents an increase in 
the grade level in the Small Purchasing Branch (SPB) at NSC 
Duarte. With all other factors remaining constant, an 
increase in the mean grade level would reflect 2 ui increase 
in inputs while output remained constant. Productivity 
would decrease in this scenario as measured by the ratio of 
outputs to inputs. 

There was an increase in the proportion of buyers to 
administrative support personnel. Buyers made up 69.9 
percent of the pre APADE work force while comprising 79.4 
percent after implementation. The mean grade level for 
buyers before APADE was GS 6.04 and after was GS 6.36. The 
mean grade level for administrative support personnel before 
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APADE was GS 4.10 and after was GS 4.55. Since there was 
not a statistically significant increase in grade level for 
buyers or administrative support personnel between the two 
periods, the overall increase in GS level in the post APADE 
period is due to the increase in the proportion of buyers 
caused by the decrease in the number of administrative 
support personnel in the work force. 

As workload decreases, one would expect to see an 
eventual decrease in the overall workforce. Although 
management would not necessarily terminate buyers and 
administrative support personnel at the same time, the 
number of personnel in both categories would be expected to 
decrease to levels in proportion to those at the pre-APADE 
levels. That was not the case in this study. APADE shifted 
some of the administrative support personnel tasks to the 
buyers. (See Appendix A) 

While the inclination would be to believe that the 
accompanying increase in grade structure would indicate a 
drop in productivity, further inspection reveals that this 
is not necessarily the case. The increase in grade 
structure is primarily due to the decrease in personnel, 
with a proportionally larger decrease in administrative 
support personnel than in buyers. Therefore, all that can 
be said about the grade structure is that APADE shifted some 
of the tasks done by administrative support personnel in the 
pre-APADE period to the buyers in the post implementation 
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period. This shift is an expected outcome with the 
application of technology into the work environment. 

3 . Overtime 

The mean amount of overtime time used per pay period 
for all personnel prior to the implementation of APADE was 
519.5 hours. The mean amount of overtime used per worker 
per pay period was 6.7 hours. The mean amount of overtime 
used per pay period for all personnel after implementation 
was 124.4 hours. The mean amount used per worker per pay 
period was 2.0 hours. This represents a decrease of 76.1 
percent in the n\amber of hours of overtime used per period, 
and a 69.8 percent decrease in the nxamber of hours used per 
person per pay period. All other factors remaining 
constant, a decrease in overtime would reflect a decrease in 
inputs, reflecting an increase in productivity. 

The post APADE management policy to use compensatory 
time in lieu of overtime is a management decision rather 
than a product of APADE. However, even if compensatory time 
earned were added to overtime, there would still be a 
decrease of 57.5 percent per person per pay period. 

B . OUTPUTS 

1 . Niomber of Purchase Orders 

The mean number of purchase orders processed during 
the year prior to the implementation of APADE was 7123 per 
month. The mean number of purchase orders processed during 
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the year after the implementation was 4992 per month. This 
represents a decrease of 29.9 percent in the number of 
purchase orders processed per month. This is primarily a 
reflection of a policy which directed previous NSC Duarte 
customers to other sources of supply. 

2 . Procurement Action Lead Time (PALT) 

The mean Procurement Action Lead Time (PALT) during 
the year prior to the implementation of APADE was 52.4 days. 
The mean PALT during the year after the implementation was 
23.5 days. This represents a decrease of 55.2 percent in 
the PALT value between the two years . All other factors 
remaining constant a decrease in PALT would indicate that 
some degree of productivity improvement has been attained. 

There was a 29.9 percent decrease in purchase 
requests at NSC Duarte between the two periods measured. 
Also, the ratio of transactions processed to the number of 
Icibor hours worked remained statistically constant between 
the two periods . This would tend to indicate that the 
decrease in PALT was primarily due to the decrease in 
workload. 

The docviment flow charts provided by NSC Duarte for 
the before and after APADE periods (Appendix A) reflect a 
significant change in the document flow process. The flow 
of documentation before APADE required 54 document movements 
between various clerks, supervisors, and procurement 
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personnel . After the implementation of APADE the new 
document flow process required only 15 document movements. 
Although APADE may not have improved the productivity of the 
personnel individually, it did streamline the procurement 
process and reduced the number of potential delays in the 
system. * 

Interviews with Small Purchasing Branch (SPB) 
personnel revealed that they believed that the 
implementation of APADE was the significant factor in the 
reduction of PALT. Although there was a 29.9 percent 
decrease in the nximber of purchase requests between the 
before and after periods, there was also a 15.0 percent 
decrease in staff size. 

We can not say with any quantitative certainty the 
extent to which APADE affected PALT. However, it can be 
said that APADE played the primary role in the reduction of 
the document flow within the procurement process. Although 
it is our opinion that PALT was dramatically reduced at NSC 
Duarte due to the implementation of APADE, no conclusive 
determination can be made to ascertain the degree of 
influence either the implementation of APADE or the decrease 
in workload has had on PALT. 



^Potential delays would be defined as time waiting in inboxes, 
outboxes, going from desk to desk, awaiting reviews, signatures, 
etc. . 
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C . SOCIAL EFFECTS 



1 . Annual Leave 

The mean amount of annual leave used per pay period 
for all personnel prior to the implementation of APADE was 
442.22 hours. The mean amount of annual leave used per 
person per pay period was 5.40 hours. The mean amount of 
annual leave used per pay period for all personnel after 
implementation was 380.05 hours. The mean amount of annual 
leave used per worker per pay period was 6.12 hours. This 
represents a decrease of 14.10 percent in the number of 
hours of annual leave taken per period and a 13.3 percent 
increase in the number of hours per person per pay period. 

The amount of annual leave used per pay period for 
all personnel is not an accurate measure of degree of annual 
leave taken in that it does not consider the fact that there 
were fewer staff members after APADE than before. 

Therefore, the appropriate measure is the leave taken per 
person per pay period which normalizes the relationship. 

All else being equal, an increase in annual leave 
taken would indicate an increase in management's inclination 
or ability to allow personnel to take leave. The increase 
in the amount of leave taken would act to decrease overall 
worker input hours, thus when taken by itself, could 
indirectly imply an increase in productivity. Because of 
the reduction in workload, no conclusive determination can 
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be made to ascertain the degree of influence either the 
implementation of APADE or the decrease in workload has had 
on annual leave . 

2 . Sick Leave 

Sick leave is a classic objective indicator of 
morale - it being the safety valve beyond management 
scheduling. The mean amount of sick leave used pay period 
for all personnel prior to the implementation of APADE was 
255.83 hours. The mean amount of sick leave used per person 
per pay period was 3.15 hours. The mean amount of sick 
leave used per pay period for all personnel after 
implementation was 262.03 hours. The mean amount of sick 
leave used per worker per pay period was 4.20 hours. This 
represents an increase of 2.40 percent in the number of 
hours of sick leave taken per period and a 33.20 percent 
increase in the nxamber of hours per person per pay period by 
SPB personnel. 

During the analysis of the sick leave used per 
worker per pay period, it was noted that the sick leave 
trend analysis remained virtually unchanged throughout the 
two periods except during one pay period in the post 
implementation period. This period included a major natural 
disaster. During this period that the amount of sick leave 
taken increased significantly. Although this disaster did 
not physically affect the APADE system, it can be inferred 
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that there was a noted increase in the stress and anxiety 
felt by workers during the period. 

Taken by itself, an increase in sick leave would 
indicate that there was a decrease in worker satisfaction. 
However, due to the environmental factors beyond the control 
of NSC Duarte's management, which accounted for a 
significant portion of that increase, the increase noted 
cannot be attributed to a decrease in worker satisfaction. 

3 . Leave Without Pay 

The mean amount of leave without pay used per pay 
period for all personnel prior to the implementation of 
APADE was 155.4 hours. The mean amount of leave without pay 
used per person per pay period was 1.87 hours. The mean 
aunount of leave without pay used per pay period for all 
personnel after implementation was 112.05 hours. The mean 
eunount of leave without pay used per worker per pay period 
was 1.80 hours. This represents a decrease of 27.90 percent 
in the number of hours of leave without pay taken per period 
and a 3.80 percent decrease in the number of hours per 
person per pay period by SPB personnel. 

This is a combined worker and management-driven 
social factor. A worker can request LWOP, but authorization 
is up to the discretion of management. All else being 
equal, a decrease in the amount of LWOP taken per worker 
could indicate a 'not unhappy' condition with the work 
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place, thus indicating an increase in worker satisfaction in 
the work environment. This would have a positive overall 
effect on productivity. In this study, there was no 
statistically significant increase/decrease in LWOP and 
therefore the only conclusion that can be drawn is that the 
implementation of APADE did not affect LWOP in the work 
environment . 

D . PRODUCTIVITY MEASUREMENT 

The mean nxamber of purchase orders processed per labor 
hour for the year prior to the implementation of APADE was 
0.5564. The mean number of purchase orders processed per 
labor hour for the year after the implementation was 0.5236. 
This represents a 5.9 percent drop in purchase orders 
processed per labor hour. Although there is a decrease in 
productivity, as measured by purchase orders processed per 
labor hour, the difference is not statistically significant. 
Therefore, the result is a productivity measurement that has 
remained constant between the two periods . 

E. ACCOMPLISHMENTS OF THE STUDY 

We applied a before and after pre-experimental design 
(Campbell and Stanley, 1966) that accumulated historical 
indicators of inputs, outputs, and social effects of office 
automation in an organization. The study of the effects of 
the installation of APADE has further established a 
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quantitative bench mark for office automation productivity. 
This was accomplished by using a standard input/output model 
of productivity. 

In this study, we also captured empirical data on 
the social impact of APADE from objective historical 
documentation rather than through a subjective user 
satisfaction survey. 



F. SUMMARY OF APADE BENEFITS 

The APADE system implemented at the Small Purchasing 
Branch (SPB) at NSC Duarte was found to enhance the 
efficiency of the procurement process in a variety of ways: 



1. Dramatically decreased the amount of document movement 
in the procurement mechanism, thus eliminating possible 
delay points in the process. 

2. Allows for automated documentation preparation, thus 
reducing the requirement for significant administrative 
(clerical) support. 

3. Provides for automated document control and tracking 
status to procurement personnel and allows customers 
access to the system to obtain status on their purchase 
requests. This frees up procurement personnel from 
having to answer phone calls from customers inquiring 
on purchase request status. 

4. Provides real time contracting information to 
management which allows them to more effectively manage 
buyer resources and distribute procurement work load. 

5. The process regulates the procurement personnel and 
work environment. Management can assign buys to 
specific buyers electronically and all purchase orders 
can be easily traced by management and buyer personnel. 
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G. DIRECTION FOR FURTHER RESEARCH 



The following is a listing of potential topics for 
further research: 



1. What has been the effect of APADE on all of the major 
supply stock points where it has been installed? Are 
the results similar to those found in this study? 

2. APADE' s primary design intent was to improve the 
buyer' s ability to make quality buys in accordance with 
established rules, regulations, and guidelines. Has 
APADE satisfied this intention? Can this type of 
information be quantitatively measured? 

3. What has been the effect of APADE on the users? Do the 
users' view APADE as a positive influence on the 
procurement process? How has user perception/attitude 
affected implementation of APADE at various sites. How 
can this type of information be quantitatively 
measured? 

4. APADE is currently being implemented in the large 
procurement process system (purchases > $25,000). A 
study is needed to measure the before and after effects 
of APADE on the large procurement cycle. 

5. A comparative study is needed on the various automated 
procurement systems currently being utilized by the 
federal government. Under what conditions would one 
system be the preferred choice over the others? Is it 
appropriate to compare systems with different designs 
on a single set of measurement criteria? 

6. A before and after productivity study on the desktop 
version of APADE currently under development for use at 
small procurement sites would be useful. 

7. A follow-up to this study on the effects of APADE at 
the NSC Duarte site is appropriate to determine if 
further improvements in the productivity are recognized 
as SPB personnel become more faimiliar with the system. 

8. A study to perform a regression analysis on those 
factors which may have had an affect on the various 
inputs, outputs, and social effects addressed in this 
study . 
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In today' s environment of diminishing resources it has 
become critical to quantitatively measure the effects of 
decisions on the organization. This has never been more 
important than it is today with automation of the white 
collar work place. 

In determining whether an organization has obtained an 
advantage or "got its money's worth" out of procuring an 
automated system, planning is a critical factor. If an 
organization has made the decision to automate or replace an 
existing system, the organization must first decide what 
they want to gain from the expenditure. 

Once the reason for the procurement has been determined, 
quantitative measures must be established to compare the 
before and after effects of the decision. The establishment 
of these measures and data collection must start early in 
the new system's development phase. Management is 
challenged with using not only common 'number crunching' 
type assessments, but also measures that can be used to 
evaluate the evolving nature of the organization. One would 
hope that quantitative and objective qualitative measures 
exist, or can be developed, to fulfill this requirement. 

During implementation of the new system, the 
measurements should continue, if nothing more than to 
observe the trend over the implementation phase and provide 
a better understanding of the dynamics of the implementation 
process. These measures should continue well into the post 
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implementation phase, to measure and note the trends in the 
growth or decline of the system. 

Unfortunately, the decision to evaluate the 
effectiveness of a system often occur after the system has 
already been implemented. At this point, it is often too 
late to get an accurate picture. Another mistake occurs 
when a system is evaluated against criteria that it was not 
designed to comply with. In the real world decisions to 
measure the 'effectiveness' of a system are made after the 
development, implementation, and a significant portion of 
the maintenance dollars have become sunk costs. 

In reviewing the literature on this s\object, it has been 
found that great expectations are given prior to, during, 
and after the development of new systems. Measurements are 
not taken until after the question arises whether or not the 
'new' system in place actually fulfilled the needs and 
requirements of the organization. 

Such is the case with this study. In conversations with 
individuals who were connected with the APADE project, it 
was realized that the system was not developed with a direct 
aim towards increasing productivity. The primary concern of 
the APADE project was to develop an environment which would 
assist contracting personnel in the compliance with existing 
contract laws and regulations and provide an automated 
climate that would evolve with the ever changing procurement 
environment . 
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There may be those who read this study and attempt to 
use it out of context to determine whether or not the U. S. 
Navy "got it's money's worth' out of the APADE system. This 
is not the intent of this study. This study's intention was 
to further develop a methodology to compare automated 
procurement systems based on specific productivity measures. 
We believe that has been done. However, additional research 
will be needed to establish a precise methodology to further 
advance this notion of quantitatively measuring the 
effectiveness of automated systems. 
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APPENDIX A: DOCUMENT FLOW 



DOCUMENT FLOW 

The diagrams that follow have been reproduced from 
original documents provided by the Small Purchase Branch at 
the Navy Field Contracting Activity where this study was 
conducted. Pages 75 through 81 represent the document flow 
prior to implementing APADE. Pages 82 and 83 represent the 
document flow using APADE. 
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APPENDIX B: SUMMARY OF INPUT BY PAY PERIOD 



PRE-APADE IMPLEMENTATION 



NUMBER OF 


AVERAGE 




EMPLOYEES 


GRADE LEVEL 


OVERTIME 


68 


5.794 


0 


68 


5.794 


0 


68 


5.794 


0 


68 


5.794 


0 


68 


5.794 


471 


70 


5.757 


368 


71 


5.718 


733 


72 


5.681 


179 


72 


5.681 


262 


73 


5.685 


565 


73 


5.685 


621 


73 


5.685 


502 


73 


5.685 


410 


74 


5.649 


720 


74 


5.649 


812 


74 


5.649 


852 


75 


5.613 


500 


75 


5.613 


926 


75 


5.613 


589 


76 


5.579 


1065 


77 


5.546 


573 


78 


5.513 


405 


78 


5.513 


828 


78 


5.513 


872 


78 


5.513 


524 


78 


5.513 


732 
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APPENDIX B: SUMMARY OF INPUT BY PAY PERIOD 



AFTER APADE 


implementation 




NUMBER OF 
EMPLOYEES 


AVERAGE 
GRADE LEVEL 


OVERT 1 1 


59 


6.068 


142 


59 


6.068 


257 


60 


6.050 


310 


61 


6.049 


523 


62 


6.048 


292 


62 


6.048 


341 


63 


6.064 


336 


63 


6.064 


165 


63 


6.064 


0 


63 


6.064 


0 


63 


6.064 


0 


63 


6.064 


0 


63 


6.064 


4 


63 


6.064 


0 


63 


6.064 


0 


63 


6.064 


0 


63 


6.064 


0 


63 


6.064 


0 


63 


6.064 


124 


63 


6.064 


506 


63 


6.064 


218 


63 


6.064 


10 


63 


6.064 


0 


63 


6.064 


0 


63 


6.064 


8 


63 


6.064 


0 
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APPENDIX C: SUMMARY OF SOCIAL EFFECTS BY PAY PERIOD 



PRE-APADE IMPLEMENTATION 



LEAVE 

WITHOUT 



PAY PERIOD 


ANNUAL LEAVE 


SICK LEAVE 


PAY 


1 


257 


383 


44 


2 


245 


286 


57 


3 


255 


355 


52 


4 


320 


150 


116 


5 


282 


258 


73 


6 


364 


277 


77 


7 


388 


350 


124 


8 


551 


241 


116 


9 


315 


169 


132 


10 


482 


242 


161 


11 


489 


174 


236 


12 


386 


274 


157 


13 


382 


241 


150 


14 


899 


188 


108 


15 


451 


215 


117 


16 


568 


259 


165 


17 


376 


256 


167 


18 


363 


192 


210 


19 


274 


194 


112 


20 


367 


318 


156 


21 


494 


252 


242 


22 


339 


320 


261 


23 


519 


423 


247 


24 


646 


223 


377 


25 


349 


232 


239 


26 


1142 


177 


134 
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APPENDIX C: 



SUMMARY OF SOCIAL EFFECTS BY PAY PERIOD 



AFTER APADE IMPLEMENTATION 



LEAVE 

WITHOUT 



: PERIOD 


ANNUAL LEAVE 


SICK LEAVE 


PAY 


1 


341 


302 


60 


2 


210 


275 


82 


3 


301 


366 


16 


4 


302 


351 


65 


5 


245 


418 


86 


6 


300 


324 


16 


7 


320 


357 


110 


8 


397 


155 


61 


9 


232 


210 


105 


10 


302 


280 


88 


11 


423 


204 


67 


12 


385 


167 


63 


13 


314 


202 


46 


14 


545 


261 


91 


15 


530 


291 


132 


16 


508 


261 


106 


17 


374 


301 


95 


18 


285 


219 


58 


19 


245 


243 


135 


20 


347 


229 


221 


21 


568 


311 


209 


22 


270 


325 


135 


23 


305 


225 


177 


24 


488 


313 


212 


25 


405 


134 


222 


26 


942 


81 


245 
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APPENDIX D: PRODUCTIVITY MEASUREMENT PER 
PRE-APADE IMPI.F.MENTATION 



PERIOD # TRANSACTIONS LABOR HOURS 



1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 



2537 

3080 

2738 

2738 

3540 

3540 

3138 

3028 

3248 

3370 

3602 

3913 

3891 

3838 

3831 

3739 

3739 

4684 

4842 

3129 

2444 

2560 

2677 

2675 

2669 

2288 



5156 

5252 

5178 

5254 

5698 

5570 

5950 

5431 

5806 

5840 

5881 

5845 

5796 

5844 

6348 

6180 

6101 

6561 

6408 

6623 

5985 

6125 

6279 

6266 

6344 

5918 



PAY PERIOD 



TRANSACTIONS 
PER LABOR 
HOUR 

0.492 

0.587 

0.529 

0.521 

0.621 

0. 636 

0.527 

0.558 

0.559 

0.577 

0.613 

0.669 

0.671 

0.657 

0.604 

0.605 

0.613 

0.714 

0.756 

0.472 

0.408 

0.418 

0.426 

0.427 

0.421 

0.387 
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APPENDIX D: PRODUCTIVITY MEASUREMENT PER PAY PERIOD 



AFTER APADE IMPLEMENTATION 



PERIOD 


# TRANSACTIONS 


LABOR HOURS 


TRANSACTIONS 
PER LABOR 
HOUR 


1 


2619 


4238 


0.619 


2 


2619 


4489 


0.583 


3 


2576 


4506 


0.572 


4 


2656 


4764 


0.557 


5 


2958 


4582 


0.645 


6 


2948 


4740 


0.622 


7 


2701 


4588 


0.589 


8 


2515 


4591 


0.548 


9 


2487 


4492 


0.554 


10 


2449 


4369 


0.560 


11 


2441 


4345 


0.562 


12 


2411 


4424 


0.545 


13 


2379 


4481 


0.531 


14 


1960 


4142 


0.473 


15 


1959 


4086 


0.479 


16 


2268 


4164 


0.544 


17 


2319 


4269 


0.543 


18 


2706 


4477 


0.604 


19 


2922 


4540 


0.644 


20 


2117 


4748 


0.446 


21 


1312 


4169 


0.315 


22 


1476 


4319 


0.342 


23 


1884 


4332 


0.435 


24 


1848 


4026 


0.459 


25 


1632 


4286 


0.381 


26 


1748 


3771 


0.464 
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